First, note that we shall use the same MDPs defined in Appendix D.1 as follows

Each client pulls an arm and communicates with neighbors based on the graph provided by the environment.

Then momentum acceleration is also introduced into our dual accelerating update step.

We study reinforcement learning in episodic inhomogeneous MDPs with adversarial full-information rewards and the unknown transition kernel. We consider the linear mixture MDPs whose transition kernel is a linear mixture model and choose the dynamic regret as the performance measure.

The proofs of Lemmas A.2 and A.3 follow from covering arguments and Bernstein's inequality; refer